CLAIMS 

What we claim: 

1. A HierarchicalSiommary Description Scheme (DS) for describing a video 
summary, the HierarchicalSummary DS comprises at least one HighlightLevel DS which is 
describing highlight level, wherein said HighlightLevel DS comprises at least one 
HighlightSegment DS which is describing highlight segment information constituting the 
summary video of the highlight level. 

2. The HierarchicalSummary DS according to claim 1, wherein said 
HighlightLevel DS is composed of at least one lower level HighlightLevel DSs. 

3. The HierarchicalSummary DS according to claim 1, wherein said 
HighlightSegment DS comprises a VideoSegmentLocator DS which is describing time 
information or video itself of said corresponding highlight segment. 4. The 
HierarchicalSummary DS according to claim 3, wherein said HighlightSegment DS further 
comprises ImageLocator DS which is describing the representative frame of said 
corresponding highlight segment. 

5. The HierarchicalSummary DS according to claim 3, wherein said 
HighlightSegment DS further comprises SoundLocator DS which is describing the 
representative sound information of said corresponding highlight segment. 

6. The HierarchicalSummary DS according to claim 3, wherein said 
HighlightSegment DS further comprises ImageLocator DS which is describing the 
representative frame of said corresponding highlight segment and SoundLocator DS which 
is describing the representative sound information of said corresponding highlight segment. 
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7. The HierarchicalSummary DS according to claim 4, wherein said 
ImageLocator DS describes time information or image data of the representative frame of 
video interval corresponding to said corresponding highlight segment. 

8. The HierarchicalSummary DS according to claim 3, wherein said 
HighlightSegment DS further comprises AudioSegmentLocator DS which is describing the 
audio segment information constituting an audio summary of said corresponding highlight 
segment. 

9. The HierarchicalSummary DS according to claim 8, wherein said 
AudioSegmentLocator DS describes time information or audio data of the audio interval of 
said corresponding highlight segment. 

10. The HierarchicalSummary DS according to claim 1, wherein said 
HierarchicalSummary DS includes SummaryComponentList describing and enumerating 
all of the SummaryComponentTypes which is included in the HierarchicalSummary DS. 

11. The HierarchicalSummary DS according to claim 10, wherein said 
SummaryComponentType includes keyFrames representing the key frame summary 
composed of representative frames, keyVideoClips representing the key video clip 
summary composed of key video segment' sets, keyEvents representing the summary of 
the video interval corresponding to either the event or the subject, keyAudioClips 
representing the key audio clip summary composed of representative audio intervals' sets, 
and unconsfraint representing the type of summary defined by users except for said 
summaries. 

12. The HierarchicalSummary DS according to claim 1, wherein said 
HierarchicalSummary DS includes SummaryThemeList DS which is enumerating the event 
or subject comprised in the summary and describing the ID and then describes event based 
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summary and permits the users to browse the surmnary video by the event or subject 
described in said SummaryThemeList 

13. The HierarchicalSummary DS according to claim 11, wherein said 
SummaryThemeList DS includes arbitrary number of SummaryThemes as elements and 
said SummaryTheme includes an attribute of id representing the corresponding event or 
subject. 

14. The HierarchicalSummary DS according to claim 13, wherein said 
SummaryTheme further includes an attribute of parentID which is to describe the id of the 
event or subject of the upper level. 

15. The HierarchicalSummary DS according to claim 13, wherein said 
HighlightLevel DS includes an attribute of themelds describing said attribute of ids of 
common events or subjects if all of the Highlights egments and HighlightLevels which are 
constituting corresponding highlight level have common events or subjects. 

16. The HierarchicalSummary DS according to claim 13, wherein said 
HighlightSegment DS includes an attribute of themelds describing said attribute of id and 
describes the event or subject of the corresponding highlight segment. 

17. A computer-readable recording medium where a HierarchicalSummary 
DS is stored therein, the HierarchicalSummary DS comprises at least one HighlightLevel 
DS which is describing highlight level, wherein said HighlightLevel DS comprises at least 
one HighlightSegment DS which is describing highlight segment information constituting 
the summary video of that the highlight level, wherein said HighlightSegment DS 
comprises VideoSegmentLocator DS describing time information or video itself of said 
corresponding highlight segment. 
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18. A method for generating video summary description data according to 
video summary description scheme by inputting original video, comprising: 

video analyzing step which is producing video analysis result by inputting 
the original video and then analyzing the original video; 

summary rule defining step which is defining the simmiary rule for selecting 
summary video interval; 

summary video interval selecting step which is constituting summary video 
interval information by selecting the video interval capable of summarizing video contents 
fi-om the original video by inputting said original video analysis result and said summary 
rule; and 

video summary describing step which is producing video summary 
description data according to the HierarchicalSummary DS by inputting the summary video 
interval information output by said summary video interval selecting step, 

19. The method for generating video summary description data according to 
claim 18, wherein said HierarchicalSummary DS comprises at least one HighlightLevel DS 
which is describing highlight level, wherein said HighlightLevel DS comprises at least 
HighlightSegment DS which is describing highlight segment information constituting the 
summary video of the highlight level, wherein said HighlightSegment DS comprises 
VideoSegmentLocator DS describing time information or video itself of said corresponding 
, highlight segment. 

20. The method for generating video summary description data according to 
claim 18, wherein said video analyzing step comprises: 

feature extracting step which is outputting the types of features and video 
time interval at which those features are detected by inputting the original video and 
extracting those features; 
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event detecting step which is detecting key events included in the original 
video by inputting said types of features and video time interval at which those features are 
detected; and 

episode detecting step which is detecting episode by dividing the original 
video into story flow base on the basis of said detected event. 

21. The method for generating video summary description data according to 
claim 18, wherein said summary rule defining step provides the types of summary events, 
which are bases in selecting the summary video interval, after defining them to said video 
summary describing step. 

22. The method for generating video summary description data according to 
claim 18, the method fiulher comprises representative frame extracting step which is 
providing the representative fi-ame to said video summary describing step by inputting said 
summary video interval information and extracting representative frame. 

23. The method for generating video summary description data according to 
claim 18, the method further comprises representative sound extracting step which is 
providing, the representative sound to said video summary describing step by inputting said 
summary video interval information and extracting representative sound. 

24. A computer-readable recording mediimi where a program is stored 
therein, the program is to execute: 

feature extracting step which is outputting the types of featiires and video 
time interval at which those features are detected; 

event detecting step which is detecting key events included in the original 
video by inputting said types of features and said video time interval at which those 
features are detected; 
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episode detecting step which is detecting episode by dividing the original 
video into story flow base on the basis of said detected key events; 

summary rule defining step which is defining the summary rule for selecting 
the summary video interval; 

summary video interval selecting step which is constituting summary video 
interval information by selecting the video interval capable of summarizing the video 
contents of the original video by inputting said detected episode and said summary rule; 
and 

video summary describing step which is generating video summary 
description data with HierarchicalSummary DS by inputting the summary video interval 
information output by said summary video interval selecting step. 

25. A system for generating video summary description data according to 
video summary description scheme by inputting original video, comprising: 

video analyzing means for outputting video analysis result by inputting 
original video and analyzing the original video; 

summary rule defining means for defining the summary rule for selecting 
the summary video interval; 

summary video interval selecting means for constituting summary video 
interval information by selecting the video interval capable of summarizing the video 
contents of the original video by inputting said video analysis result and said summary rule; 
and 

video summary describing means for generating video summary description 
data with HierarchicalSummary DS by inputting the summary video interval information 
output by said summary video interval selecting means. 

26. The system for generating video summary description data according to 
claim 25, wherein said HierarchicalSummary DS comprises at least one HighlightLevel DS 
which is describing highlight level, wherein said HighlightLevel DS comprises at least one 
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HighlightSegment DS which is describing highlight segment information constituting the 
summary video of the highUght level, wherein said HighlightSegment DS comprises 
VideoSegmentLocator DS describing time information or video itself of said corresponding 
highlight segment. 

27. The system for generating video summary description data according to 
claim 25, wherein said video analyzing means comprises: 

feature extracting means for outputting the types of features and video time 
interval at which those features are detected by inputting the original video and extracting 
those features; 

event detecting means for detecting key events included in the original 
video by inputting said types of features and video time interval at which those features are 
detected; and 

episode detecting means for detecting episode by dividing the original video 
into story flow base on the basis of said detected event. 

28. The system for generating video summary description data according to 
claim 25, wherein said summary rule defining means provides the types of summary 
eyents, which are bases in selecting the summary video interval, after defining them to said 
video summary describing means. 

29. The system for generating video summary description data according to 
claim 25, the system further comprises representative frame extracting means for providing 
the representative frame to said video summary describing means by inputting said 
summary video interval information and exfracting representative frame. 



30. The system for generating video summary description data according to 
claim 25, the system fiirther comprises representative sound extracting means for providing 



the representative sound to said video summary describing means by inputting said 
summary video interval information and extracting representative sound. 

31. A computer-readable recording medium where a program is stored 
therein, the program is for functioning: 

feature extracting means for outputting the types of features and video time 
interval at which those features are detected; 

event detecting means for detecting key events included in the original 
video by inputting said types of features and said video time interval at which those 
features are detected; 

episode detecting means for detecting episode by dividing the original video 
into story flow base on the basis of said detected key events; 

summary rule defining means for defining the summary rule for selecting 
the summary video interval; 

summary video interval selecting means for constituting summary video 
interval information by selecting the video interval capable of summarizing the video 
contents of the original video by inputting said detected episode and said summary rule; 
and 

video summary describing means for generating video summary description 
data with HierarchicalSummary DS by inputting the summary video interval information 
output by said summary video interval selecting step. 

32. A Video browsing system in a server/client circumstance, comprising: 
a server which is equipped with video summary description data generation 
system which generates video summary description data on the basis of 
HierarchicalSummary DS by inputting original video and links said original video and 
video summary description data; and 
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a client which is browsing and navigating video by overview of said original 
video and access to the original video of said server using said video summary description 
data. 



28 



